[SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page by itholic · Pull Request #32658 · apache/spark

itholic · 2021-05-25T07:17:02Z

What changes were proposed in this pull request?

This PR proposes move CSV data source options from Python, Scala and Java into a single page.

Why are the changes needed?

So far, the documentation for CSV data source options is separated into different pages for each language API documents. However, this makes managing many options inconvenient, so it is efficient to manage all options in a single page and provide a link to that page in the API of each language.

Does this PR introduce any user-facing change?

Yes, the documents will be shown below after this change:

"CSV Files" page

Python

Scala

Java

How was this patch tested?

Manually build docs and confirm the page.

SparkQA · 2021-05-25T08:31:16Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43434/

SparkQA · 2021-05-25T09:10:59Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43434/

docs/sql-data-sources-csv.md

SparkQA · 2021-05-25T11:57:48Z

Test build #138912 has finished for PR 32658 at commit 5ba96ec.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-26T08:33:33Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43489/

docs/sql-data-sources-csv.md

SparkQA · 2021-05-26T10:05:02Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43489/

SparkQA · 2021-05-26T11:51:23Z

Test build #138974 has finished for PR 32658 at commit 0e75d8d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

docs/sql-data-sources-text.md

docs/sql-data-sources-csv.md

SparkQA · 2021-05-27T03:27:17Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43521/

SparkQA · 2021-05-27T04:02:47Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43521/

SparkQA · 2021-05-27T04:34:59Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43523/

SparkQA · 2021-05-27T05:08:57Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43523/

SparkQA · 2021-05-27T07:00:09Z

Test build #139003 has finished for PR 32658 at commit 673e26b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-27T07:51:25Z

Test build #139005 has finished for PR 32658 at commit 9df317f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

docs/sql-data-sources-csv.md

SparkQA · 2021-05-28T05:09:31Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43561/

SparkQA · 2021-05-28T05:15:37Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43562/

SparkQA · 2021-05-28T05:23:28Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43564/

SparkQA · 2021-05-28T05:58:09Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43564/

SparkQA · 2021-05-28T06:00:25Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43560/

SparkQA · 2021-05-28T08:12:19Z

Test build #139042 has finished for PR 32658 at commit 1991031.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-28T09:18:20Z

Test build #139044 has finished for PR 32658 at commit a4a2bb2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

itholic · 2021-05-31T03:55:37Z

@itholic the generated doc looks a bit weird:

It includes # noqa

Can you double check and fix? Seems like we should fix other places for JSON, etc.

Thanks, @HyukjinKwon . Just fixed them in every place.

SparkQA · 2021-05-31T04:39:34Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43614/

SparkQA · 2021-05-31T04:41:41Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43614/

docs/sql-data-sources-csv.md

SparkQA · 2021-05-31T08:10:15Z

Test build #139093 has finished for PR 32658 at commit 3bda8db.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

docs/sql-data-sources-csv.md

HyukjinKwon

LGTM otherwise.

HyukjinKwon · 2021-06-01T01:58:45Z

Merged to master.

SparkQA · 2021-06-01T02:40:39Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43646/

SparkQA · 2021-06-01T03:16:32Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43646/

SparkQA · 2021-06-01T06:38:22Z

Test build #139126 has finished for PR 32658 at commit f55a2fa.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

itholic changed the title ~~[SPARK-35433][DOCS] Move CSV data source options from Python and Scal…~~ [SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page May 25, 2021

github-actions bot added CORE DOCS PYTHON SQL STRUCTURED STREAMING labels May 25, 2021

HyukjinKwon reviewed May 25, 2021

View reviewed changes